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Abstract 

Nowadays, in modern agriculture, the combination of image processing techniques and intelligent methods has been 

used to replace humans by smart machine. There is no international or domestic standard classification for Parus variety. 
In this study, an artificial image processing and artificial neural network (ANN) method was used to classify strawberry 
fruit of Parus variety. In the first step, the fruit was divided into 6 classes (ANN outputs) by the expert, and 100 samples 
were randomly collected from each class. In the next step, the images of the samples were captured and three geometric 
properties with twelve color properties (as ANN inputs) were extracted. Morphological and color characteristics of the 
area of the fruit were extracted using the functions defined in Matlab software. Optimum artificial neural network 
structures (15-18-6) considering root mean squared error (RMSE) and correlation coefficient (R?) were investigated to 
classifiy the strawberry samples. Finally, the perceptron neural network with a structure of 6-18-15 was selected with the 
classification rate of 83.83%. The results obtained by ANN showed that the lowest and highest accuracy was related to 
small ripe (about 65%) and small raw (about 100%). Based on the accuracy of the results and economic aspects, ANN 
method is considered a proper method to classify strawberry fruit. 
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Introduction harvest waste of the strawberry fruit is very 
Strawberry is one of the finest and native high due to the use of poor quality processing 
fruits with delicious taste which cultivated in systems, lack of knowledge and skills in 
moderate region. Due to medicinal and producing and_ harvesting, inappropriate 
nutritional properties the cultivation of postharvest mechanisms and activities 
strawberry is increasing (Nezmer et al., 2018). associated with marketing channels (Salami et 
This fruit is a rich source of vitamin C, al., 2010). 
minerals, and other ingredients which are Quality is a human concept that incorporates 
essential in the body (Samimi and Khodaei, many parameters (Nemzer et al., 2018). The 
2011). Typically, usually consumed raw and quality of the fruit is defined by various 
fresh, but also used in preparation of jellies, parameters. According to various 
jams, ice creams and sweets (Taghavi, 2005). investigations, data such as color, shape, size 
Iran with cultivation land of more than 3900 and mass they are not sufficient for the 
hectares of strawberries is the twentieth in the consumer to buy a product; they would like to 
world among the countries which produce have more information about the fruit that is 
strawberries. The average yield of strawberries supposed to be consumed (Voca et al., 2008; 
is 14 tons per hectare which has a good growth Doving and Mago, 2002). For example, in the 
related to the European countries (FAO, 2017). case of strawberries, the quality is depending on 
According to the reports of agricultural the appearance of the fruit (the intensity and 
organization of Iran, more than 25% of total distribution of the color, the shape, size, non- 
strawberries production lost during the destruction factor), its rigidity and its taste (as 
harvesting until marketing. The amount post- measured by the amounts of sugars, organic 
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acids, phenolic acids and aromatic compounds). 
By increasing the content of the acid and 
increasing the amount of vitamin C, the soluble 
solids and the sugar of the fruit, the taste will be 
improved (Mitcham, 1996). Qualitative grading 
of various agricultural products can be effective 
in reducing post-harvest damage, maintaining 
quality and improving their commercial value. 
Strawberries are no exception to this rule. In 
recent years, attention has been focused on non- 
destructive methods for the qualitative grading 
of agricultural products. One of the common 
techniques is machine vision technology, which 
combines image processing techniques and 
intelligent methods. This technology has 
advantages over traditional and malicious 
methods such as no need for expert manpower, 
more precision, lower cost (Chen et al., 2002), 
quick response, identifying several indicators 
simultaneously and non-destructive (Liming 
and Yanchao, 2010). This technology can help 
to identify the apparent imperfections of the 
product during various process (Riquelme et 
al., 2008), Qualitative Grading (Leemans and 
Destain, 2004) and the product classification 
based on shape and size (Abdullah et al., 2006). 

Several studies have been carried out on the 
use of machine vision in grading and 
classification of agricultural products. In order 
to cluster the strawberries, a technology of 
machine vision was used to recognize the 
number of features including the size (the 
largest diameter), shape (the edge of the fruit) 
and color (the a-channel) considering k-means 
algorithms. The accuracy of the clustering 
process using this algorithm was about 95%, 
90%, 88% for the characteristics of the largest 
diameter, edge of the fruit and a-channel of 
color, respectively (Liming and Yanchao, 
2010). It was also used to classify the raisin 
product on four grades: green without tail, black 
without tail, green with tail and black with tail. 
The algorithm was based on image processing 
and intelligent methods. The superior features 
of the raisin shape and color were selected after 
segmentation of the images by Otsu method 
(Otsu, 1979) using the Correlation-based 
Feature Selection which acts on the basis of the 
correlation of the selected properties. The 


classification was also carried out using backup 
vector machine, decision tree and artificial 
neural network, which was the most suitable 
method with accuracy of 96.33% (Mollazade et 
al., 2012). In another research, almond image 
processing techniques were classified into three 
classes as: large, medium and small, based on 
the image area. After segmentation the images 
of the specimens, their area was measured. 
Then the weight of the samples was measured 
using a balance and it was selected as the basis 
for class selection with the accuracy of 99.70% 
(Castelo-Quispe ef al., 2013). In a study on the 
classification of strawberry fruit (not a specified 
variety) a new method without considering 
ANN method with accuracy of 0.94-0.97 was 
performed and the consumed time to define the 
class of the strawberry was about 0.4-0.55 s for 
each strawberry (Mar Oo and Zar Aung, 2018). 
Other methods were also used to grading the 
strawberry fruit (Feng et al., 2008; Nagata et 
al., 2000). Moreover other researchers used 
ANN method for classification of the fruits 
such as pineapple (Chia ef al., 2012), kiwi 
(Torkashvand et al., 2017) and apple (Singh et 
al., 2015) based on different parameters. 

It was found that the combination of image 
processing techniques and computational 
intelligence is a reliable method to classify the 
agricultural products. Thus, in the present 
study, the processing techniques and also 
artificial neural network were used to classify 
the strawberry of Parus variety. Moreover, 
strawberry classification was carried out using 
artificial neural network as an input data to the 
system and several features were identified for 
this purpose. Also unlike the other studies in 
producing the images from the samples, the 
fruit flower bowl was not removed to maintain 
its quality. 


Materials and methods 

ANN structure: 

An artificial neural network contains three 
layers including input, output and processing 
layers. Each layer has a group of nerve cells 
(neurons) that generally associated with all the 
neurons of the other layers, except the user 
limits communication between neurons; but 
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this is the main point that the neurons in each 
layer do not correlate with other neurons in the 
same layer. 

For the nervous cell c, we designate the input 
from the neural cell p to this by, cell. According 
to Schmidhuber, 2015 the weight of this input 
iS Wpc, and we designate the sum of the 
multiplications of inputs with their weights by 


ac: 
(1) 
= > Wye X Dye 
p 


Now, the nonlinear function has to be 
applied to a, and the output indicated by be. 


be = 9-(a¢) (2) 


Similarly, the output from the nervous cell c 
enters the n cell named ben and its weight by 


Wen. Now we put all the weights of this neural 
network into a set called W to learn the weights 
to the network. Finally, by considering x as 
input and y as output data, the following 
equation obtained which must be minimized: 


Q(W) = Uy Ged-¥i) 3) 


For the present study, first strawberry fruit 
samples were collected from Kurdistan 
province, Iran. Samples were kept in a lighting 
enclosure and after cutting fruit, images from 
the background the characteristics were 
extracted. Finally, the samples were classified 
using artificial neural network as shown in 
algorithm which depicted in Fig. 1. 


Separating the 
fruit from the 
field and the => 
sepal, receptacle 
and peduncle 


Taking pictures 
of strawberry 


Preparation of 


strawberry Classification 


of images 


Extracting 
Features ad 


samples fruit 


Fig. 1. The main steps of the proposed algorithm to classification 


The number of neurons for the hidden layer 
was determined using the method of trial and 
error. The hidden layer is important for the 
ANN modeling process and it is directly 
affected on the performance of the collector. 
About 30 times run was performed to define the 
best structure based on output data and the 
relative error. 


Collection and classification of samples 

In Kurdistan province, strawberries are often 
cultivated in open area and the land under 
cultivation of Parus variety is surpassing other 
varieties due to the various reasons. Therefore, 
in the present study, Parus strawberry fruit 
samples were collected from a strawberry farm 
located in Noshur Sofla, a village of Kamyaran 
city, Kurdistan province, Iran in Spring 2014. 
Samples were transferred to the laboratory of 
agricultural machinery Department, University 


of Ilam. During the transportation process, no 
changes in color and texture were observed in 
the fruit. 

Considering the other published papers 
related to the strawberry classifications, it was 
found that there was no international or 
domestic standard for Parus _ variety 
classification. Therefore, according to the 
expert's opinion and the characteristics of the 
size, shape and color, the samples were divided 
into six classes (Fig. 2). In order to verify the 
accuracy of the classification, some physical 
properties (geometric dimensions, mass, 
volume, apparent and actual density, porosity), 
mechanical properties (rigidity and 
bioavailability), and chemical properties 
(ascorbic acid, acidity, total soluble solids and 
flavor index) were considered as summarized 
in Table 1. 
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Fig. 2. Different classes of strawberry (Parus variety) used for this research: 1) small ripe, 2) medium ripe, 3) 
big ripe, 4) small raw, 5) medium raw, 6) big raw. 


Table 1. The Strawberry properties 


i Class 

Properties 1 2 3 4 5 6 
Height (mm) 29.44 36.63 42.48 26.47 33.08 42.07 
Mass (gr) 4.925 11.909 21,779 5.471 11.71 21.437 
Apparent Density (kg/m*) 942.367 962.449 970.166 860.592 861.623 881.513 
Actual Density (kg/m*) 561.743 430.873 401.441 498.408 385.318 376.956 
Porosity 40.39 55.232 58.621 42.085 55.28 57.238 
Rigidity (N/m) 10.1 7.8 a2 28.2 13.7 11.5 
Bioavailability (N) 13.7 16.9 297 74.5 50.1 35.1 
Ascorbic Acid (10° gr/gr) 39.53 44.06 47.2 123 TZ 13.55 
Acidity 0.55 0.64 0.67 1.08 1.1 0.99 
Total Soluble Solids (%) 8.9 8.6 8.4 4 4.2 5.5 
Flavor Index 16.3 1 12.6 3.6 3.8 3.0 
Images preparation color was selected. To capture images, 


About 100 images of each class were 
selected randomly for production of images. To 
remove environmental noise, an image 
processing box of 30*30*30 cm? was used. 
Tubular LED bulbs were also used inside the 
box to provide the best lightning quality and 
low disturbed noise on images. To eliminate the 
effect of the noises and the shadows, the black 


htning lampy 


Samsung Galaxy S II with 8 megapixels 
(3248x2448 pixels) resolution camera was used 
which placed at a distance of 30 centimeters 
above the samples. For lightening D65 as 
standard value was considered for the process. 
The box made for the image processing and the 
lightning illustrated in Fig. 3. 


Fig. 3. The image processing box 
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Images segmentation 

Image segmentation is one of the most 
important parts of image processing method. It 
is divided into pieces or constructive objects 
and there are several ways to segment the 
image. In this research, thresholding method 
was used using R, G, B color components in 
RGB color space to separate strawberry fruit 
from the background and flower bowl. After 


separating the strawberry fruit from the 
background and the flower bowl, a 
morphological operation of the dilatation and 
wear with a circular structure (using open and 
closed operators) was used to repair the edges 
of the strawberry fruit. Fig. 4 shows the various 
steps undertaken in the image processing 
(Gonzalez and Wood, 2018). 


Fig. 4. Image segmentation: a) Sample image in RGB color space, b) Deletion of background, c) Deletion of 
sepal, receptacle and peduncle, d) Filling the holes and repairing the image border, e) Isolated fruit after 
segmentation. 


To separate the fruit and bowl from the black 
background, three red, green and _ blue 
components were first extracted. A gray image 
was defined using the difference between the 
red and blue color components. In this image, 
the border between the fruit and the bowl! with 
the background was visible. The threshold 
function with defined value in this boundary 
was applied to the image and the binary image 
was obtained with values of 1 for fruit and 
flower bowl and zero for background. By 
applying the opening and closing functions and 
their combination, bumps, troughs and pits 
were modified. After that by combination of 
three red, green and blue components of the 
original image, the color image of the fruit and 
bowl separated from the background 
(considering the binary image had zero value) 
was obtained. To separate the bowl from the 
fruit the similar steps were taken, except that 
the linear combination of all three color 
components was used to obtain the gray image. 


Morphological and color characteristics of the 
area were extracted using the functions defined 
in Matlab (R2012a). 


Extraction of fruit features 

Many features have been developed on 
image processing issues. The purpose of 
extracting image attributes is to describe the 
image based on some statistical descriptors. 
The features are categorized into three 
categories: morphological or geometric 
characteristics, colors and texture. Each of these 
features could be used for various applications, 
including morphological characteristics 
(diameter, environment, area, shape and 
volume) used in grading crops, color 
characteristics used to determine the level of 
handling, breakdowns and_ spoilage. The 
appearance of agricultural products and tissue 
features are used to determine the level of 
shrinkage of agricultural products and the 
recognition of plant diseases. In this study, 
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fifteen 


characters, 
morphological characteristics and twelve color 
a 


including three 


characteristics, were used to classify the 
strawberry fruit of Parus (Table 2). 
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Fig. 5. A plot of Matlab environment applied for image processing of strawberry fruit 
Table 2. The Features extracted from the images 
Color 
Name Name Apparent feature 
feature 

Fl MeanR F13 Area 

F2 MeanG F14 Perimeter 

F3 MeanB F15 AspectRatio 

F4 StdR 

F5 StdG 

F6 StdB 

F7 SkewnessR 

F8 SkewnessG 

F9 SkewnessB 

F10 KurtosisR 

F11 KurtosisG 

F12 KurtosisB 

To calculate the morphological main diameter to the mean diameter of the 


characteristics, the images were first converted 
to the binary images, and then three properties 
of the area, the environment and the ratio of the 


samples were extracted. Due to the distinction 
between different classes in terms of shape and 
size, the morphological features were used for 
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classification process. By examining different 
color spaces, the RGB color space was chosen 
because of the difference in color components 
at different classes. Four statistical descriptors 
related to the color including: mean, standard, 
skewness and kurtosis were extracted from 
strawberry samples. MATLAB - software 
(R2012a) was used to plot the images and 
extract morphological features of color (Fig. 5). 

R, G and B represented red, green, and blue 
components of the fruit image separated from 
the background and the bowl, respectively. 
Other parameters are presented by Eqs. 4-7 as 
follow 


Mean = =D, x; (4) 
Std= |= YE i(% — ») (5) 
yn (-p3 
Skewness = eee (6) 


Inputs or input layer neurons 


Hidden layer neurons 


lun 4 

Kurtosis = eee a (7) 

ANN setting 

In order to classify strawberry fruit, the 
artificial neural networks (ANNs), one of the 
methods of data mining, was used (Omid et al., 
2009). Each ANN structure consists of at least 
three layers of the neuron: 1. Input layer, 2. 
hidden layer, 3. Output layer. The number of 
neurons in the input, hidden, and output layers 
were identified as the number of inputs vector 
variables, the difficulty and complexity of the 
classification and the number of output classes, 
respectively. As shown in Fig. 6, the number of 
neurons in the input layer is equal to the number 
of properties extracted from the sample images 
(15 characteristics), and the number of output 
layer neurons is also considered equal to the 
number of grades. The model with the structure 
of 15-18-6 was concluded as the suitable. 


Output layer neurons Outputs 


Small ripe 


Oa Z 


Fig. 6. ANN structure used for classification 
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It should be noted that there is always a 
tendency toward the lowest number of layers 
and neurons in the system. This reduces the size 
of the network and increases the speed of 
training which is very important in online 
studies. Other important factors in ANN design 
are activity function, training algorithm and 
number of training rounds. In this study, 
artificial neural network with multiplayer 
perceptron structure was used. This structure is 
appropriate for online structural applications, 
because there is no need for a timed order for 
input data and the input neuron activity function 
is linear. Also, in order to increase the learning 
speed and solve the complexity of the problem, 
the linear sigmoid activity function was used in 
the hidden layer neurons and the output layer. 
Training was also used bundled with 
momentum learning algorithm with momentum 
of 0.7 and learning factor of 0.6. This algorithm 
is one of the fastest learning algorithms which 
varies by varying the amount of momentum, 
changing the learning speed to match the 
inconsistency or incompatibility of input data, 
which results in the deletion of data from the 
learning process. Also in this algorithm, the 
stop index is the increase in the mean square 
error (MSE). Six hundred paired vectors were 
assigned to the network, of which 50% were 
used for training, 20% for assessment, and 30% 
for testing. ANN modeling was done in 
NEUROSOLUTION 5 software. 


Evaluation of the results 

In order to evaluate the ANN performance in 
classification of Parus strawberry fruit and to 
select the best structure of statistical indexes, 
classification rate (CCR (%)), correlation 
coefficient (r) and root mean square error 
(RMSE) were used in different training periods 
(Eqs. 8 and 9): 


n 
1 2 
RMSE = aD Ca¥) (8) 
t=1 


- Yiei(%, ~ Xp) 


r=1 
ae 0) 


Results and discussion 

Selecting the best ANN structure 

After implementing different ANN 
structures, the best structure was selected based 
on correlation coefficient (Fig. 7) and root mean 
square error (Fig. 8). The hidden layer which 
makes the model predict the class of the 
samples directly affected on the accuracy of the 
classification process. Fig. 7 shows that the 
pattern of the curve plots generated by ANN 
and the actual data are the same. The values of 
the correlation coefficient and root mean square 
error for the number of hidden neurons (2 to 20) 
and the number of different training periods 
(100, 300, 500, 750 and 1000 rounds) were 
investigated. It is clear from Figs. 7 and 8 that 
with increasing the number of hidden neurons 
and increasing the number of periods, the 
correlation coefficient increases as well. But the 
root mean squared error decreases, among 
which the single-layer perceptron structure with 
eighteen neurons in the hidden layer (with 
function linear sigmoid activity and momentum 
training algorithm with values of learning 
coefficient of 0.6 and momentum of 0.7) are 
less correlated to other structures with a higher 
correlation coefficient (0.83) and root mean 
square error (0.20). 


ANN strawberry classification 

After selecting the best structure of ANN, it 
was used for classification of strawberry fruit of 
Parus variety in six categories including: small 
ripe (first class), middle ripe (second class), big 
ripe (third class), small raw (fourth class), 
medium raw (fifth class) and big raw (sixth 
class). Also, in order to verify the network 
performance, the confusion matrix of the 
network (Table 3) was calculated and the 
classification rate (Table 4) was extracted. 

The confusion matrix is a square matrix that 
the number of rows and columns is equal to the 
number of classes. The elements of the i-th 
column represent the i-class of the samples 
which classified by artificial neural network 
into different classes in the rows. Moreover, the 
classification rate shows the accuracy of the 
ANN in classifying the samples in each class. 
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For example, the i-class classification rate is in the main diagonal are 100% and the others 
obtained by dividing the aii element by the sum are 0%. The elements in Table 3 show that the 
of i-class elements. In fact the optimal matrix is closer to diagonal matrix. 

confusion matrix is the one which the elements 


al] —?— 1000 epoch —8— 750 epoch —*— 500 epoch —®— 300 epoch —— 100 epoch 
0.89 - 
0.69 - 
0.49 5 
0.29 + 


0.09 - 


-0.11 0 


-0.31 ; 


Correlation Coefficient (r) 


-0.51 5 


-0.71 5 


-0.91 - 


-1.11 - 


Number of hidden layer 


Fig. 7. Variations of Correlation coefficient with the number of hidden layer neurons and number of training 
courses 
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0.29 - 
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0.25 - 


0.23 + 


Root mean squared error (RMSE) 


0.21 - 


0.19 1 
0 2 4 6 8 10 12 14 16 18 20 


Number of hidden layer 


Fig. 8. Variations of RMSE withthe number of hidden layer neurons and number of training courses 
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Table 3. The confusion matrixof the network based on the number of output classes of ANN to validate the 
classification process ranged between 0-100 % 


ANN Real 
ResaiCHOn Small ripe Medium ripe Big ripe Small raw Medium raw Big raw 
Small ripe 65 4 0 0 0 0 
Medium ripe 35 90 12 0 0 2 
Big ripe 0 4 81 0 0 13 
Small raw 0 0 0 100 8 0 
Medium raw 0 0 0 0 84 2 
Big raw 0 2 7 0 8 83 
Table 4. The classification rate for strawberry fruit 
Statistical Rel 
index Small Medium peas Small Medium ; 
: ° Big ripe Big raw 
ripe ripe raw raw 
CCR (%) 65 90 81 100 84 83 
The values of classification rate of precision was related to the small raw class 


strawberry fruit (Parus variety) are listed in 
Table 4. Based on the turbulence confusion 
matrix, the lowest accuracy of the strawberry 
classification was related to the small class 
(first class) which was about 65% due to the 
similarity of this class with the middle class. It 
can be implied that about 35% of the samples 
(in the first class) were mistakenly classified in 
the second class. As shown in Fig. 1, only the 
morphological features of these two classes can 
be distinguished, while color-based features do 
not make any distinction between the 
mentioned classes. In this study, during 
preparing the samples, the top section of the 
strawberries was not separated from the fruit 
because it has significant impact on maintaining 
the fruit quality. In the other hand by removing 
the top section of the fruit the spoilage rate 
increases. Therefore in image processing 
operations, to remove the top section even by 
applying image restoration techniques some 
parts were lost or eliminated from the texture of 
the fruit. This leads an error in the calculation 
of morphological properties which affected on 
the classification process. Also, a significant 
number of the first class specimens were 
located at the boundary between the first and 
second class, morphologically. The highest 


(fourth grade) with the accuracy of 100%. This 
was due to the entire class distinction with the 
other classes based on morphological and color 
characteristics. The accuracy of the 
classification in the other columns revealed that 
the classification accuracy was between 80% 
and 90%. Other researcher’s studies on the 
classification process of fruit reported the same 
results (Hameed et al., 2018). The factor that 
may reduce classification accuracy in this study 
is the lack of standard classification and 
presence the large number of samples on the 
boundary between the two classes defined by 
the expert. Also the large number of classes can 
be another factor for reducing the accuracy of 
the network. In this research, the classification 
rate was about 83.83 %, which shows that the 
qualitative grading of strawberry of Parus 
variety based on morphological and color 
characteristics using ANN methods can be very 
applicable especially in the design of grading 
and sorting machines. The results obtained by 
the present study are in agreements with the 
results obtained for the classification of the 
strawberry (Nagata et al., 2000; Feng et al., 
2008; Liming & Yanchao, 2010; Mar Oo and 
Zar Aung, 2018). 
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Comparison between the data obtained 
experimentally and by the software showed that 
the time taken to classify the one hundred of 
strawberries by an expert is about 117 s while 
for the software it was about 2.5 s. In addition, 
if the erosion and fatigue factors of the expert 
are considered, then the classification time is 
increased. For a farmer maximizing the harvest 
or post harvest process with low price is 
considered as the main goal especially for the 
strawberry which the different conditions such 
as weather, atmospheric conditions and 
workers skill directly affecting the yield. Thus 
based on obtained results of ANN method 
applied to classification of Parus variety, and 
economic considerations it can be implied that 
ANN method is a suitable method used for 
grading strawberry fruit. 


Conclusion 

One of the main stages of harvesting and 
post harvesting processes is classification of 
food materials produced in agricultural sector. 
Parus variety is the popular strawberry planted 
in the west of Iran, especially Kurdistan 
province. Artificial neural network as a fast and 
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